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Abstract. Divergences often play important roles for study in information science so that 
it is indispensable to investigate their fundamental properties. There is also a mathematical 
significance of such results. In this paper, we introduce some parametric extended divergences 
combining Jeffreys divergence and Tsallis entropy defined by generalized logarithmic functions, 
which lead to new inequalities. In addition, we give lower bounds for one-parameter extended 
Fermi-Dirac and Bose-Einstein divergences. Finally, we establish some inequalities for the Tsallis 
entropy, the Tsallis relative entropy and some divergences by the use of the Young's inequality. 
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1 Introduction 

For the study of multifractals, in 1988, Tsallis |27j introduced one-parameter extended entropy 
of Shannon entropy by 

n n ^ 

Hgip) ^-Y^p] ln,p,- = Y,Pj l'^'? -' (9 > 0, g / 1) (1) 

j=i j=i Pi 

where p = {pi,P2, ■ ■ ■ ,Pn} is a probability distribution with pj > for all j = 1, 2, • • • , n and 
the g— logarithmic function for x > is defined by Ing(x) = ^^il^^ which uniformly converges 
to the usual logarithmic function log(x) in the limit g — )■ 1. Therefore Tsallis entropy converges 
to Shannon entropy in the limit q ^ 1: 

n 

lim H, (p) = (p) = - ^ log pj. (2) 



It is also known that Renyi entropy |18j 

R,{p)^-—iog{ypU (3) 
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is one -parameter extension of Shannon entropy. 

For two probability distributions p = {pi,p2,-" ^Pn} and r = {ri,r2,-- - ,rn} we have 
divergences based on these quantities ([T|) and ([3|). We denote by 

n n 

^<?(p||r) = ^p'jiln^Pj -lugrj) = -^p^lug^ (4) 

Tsahis relative entropy. Tsallis relative entropy converges to the usual relative entropy (diver- 
gence, Kullback-Leibler information) in the limit g — >• 1: 

n 

lim-Dg(p||r) = i:>i(p||r) = Vpj(logpj -logr^). (5) 

i=i 

We also denote by i?q(p||r) the Renyi relative entropy [18j defined by 

(6) 



i?,(p||r) = -l^log I^X^pJrj 



Obviously limq_>i i?g(p||r) = £'i(p||r). 

The divergences can be considered to be a generalization of entropies in the sense that Shan- 
non entropy can be reproduced by the divergence logn — Di(p||u) for the uniform distribution 
u = {1/n, 1/n, ••• ,1/n}. Therefore the study of divergences it is important for the develop- 
ments of information science. In this paper, we study several mathematical inequalities related 
to some generalized divergences. 



2 Two parameter entropies and divergences 

In this section and throughout the rest of the paper we consider p = {pi,P2,''' ,Pn} and 
r = {ri, r2, • • • , r„} with pj > 0, rj > for all j = 1, 2, • • • ,n to be probability distributions. 

We start from the Tsallis quasilinear entropies and Tsallis quasilinear divergences as they 
were defined in llOl. 



Definition 2.1 ([10]) For a continuous and strictly monotonic function ip on (0, oo) and r > 

with r 7^ 1 (the nonextensivity parameter), Tsallis quasilinear entropy (r- quasilinear entropy) is 
defined by 



(7) 



In this context, as a particular case of Tsallis quasilinear entropy we have Sharma-Mittal 
information measure (|16j.|20j.|21j). that is for ■0(x) = x^~'^ we have: 



if "(p) = In^ 




1 



1 



S-M 
r,q 



(P)- 



^p) = Hq{p). Sharma-Mittal entropy is also seen in the literature as a 



We find that Ff^*^ 

two-parameter extension of Renyi entropy [19\ Section 5] . This also gives rise to another case of 
interest 



1 

1-9 



I|^(p)=ln2^ \^J2p] 
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(8) 
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which coincides with Arimoto's entropy for q = 1//3, cf. [3j, and with i?-norm information 
measure, for i? = g^, cf. [4]. 

Definition 2.2 (|10j) For a continuous and strictly monotonic function tp on (0, oo), the 
Tsallis quasilinear relative entropy is defined by 



Dfip\\r)^-lnri;-'\Y,Pji^[- 

j=i Vfj, 



(9) 



Sharma-Mittal divergence ([2],[1^) becomes now a particular case of Tsahis quasihnear di- 
vergence: 



1-9 



1 

1-q 



1 

1-q 



1-r 



1 



In, I J^pH 

71 q 1— g 1 1-9 



^-[E]=iPh 



1 — r 1 — r 

By anafogy to the entropy computation, we find the following Arimoto type divergence: 



^tl(pllr) 




(10) 



Remark 2.3 In limit r — t- 1 we have I^^ '(p) — )■ /f^ ''(p) = Rq{p) and Df^ ''(p||r 



1-9 / 



Df^ '(pllr) = i?g(p||r). It is known that for q ^ r the Sharma-Mittal divergence fails to conform 



to Shore- Johnson theorem 123[ 24. \25^. that is Sharma-Mittal divergence cannot be written as a 
f— divergence 



for some function f. The previous limits give us a very intuitive way to conclude that Renyi 
divergence has a similar failure 12]. Also this enables us to say that the two-parameter extended 
relative entropy discussed in /5, Section 6] cannot be seen as a particular case of Sharma-Mittal 
divergence. 



Remark 2.4 For a; > and r > with r ^ 1, we define the r- exponential function as the inverse 
function of the r -logarithmic function by exp,(x) = {1 + (1 — r)x^^^^'^^''\ z/ 1 + (1 — r)x > 0, 
otherwise it is undefined. Here is another connection among Sharma-Mittal entropy, Tsallis 
entropy and Renyi entropy U0^ : jl22\, (B.8)]: 

exp^H^-^{p) = exp^Hgip) = expRg{p). 

As for a connection among their divergences, we get 

exp2.. «-*^(p||r)) = {1 + (r - l)D.^-^^(p||r)}^/('-^) = {1 + (g - l)D,{p\\r)y/^'^~'^ 

= exp2_(; {Dq{p\\r)) = expi?5(p||r). 
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Remark 2.5 The weighted quasilinear mean for some continuous and strictly monotonic func- 
tion i/; : / — )• M is defined by 



M[^]{xi,X2,- ■ ■ ,Xn) ^ , (11) 

where Y2]=iPj — 1; Vj > 0, Xj G / for j = 1,2,--- ,n. It is known that Af[^] (xi, X2, • " " ^^n) = 
Af[(p] (xi, X2, • ■ ■ jXn) if and only if ip and ip are affine maps of each other, i.e. there ex- 
ist constants a, b such that tp = aip + h (cf. JT, page I4I], cf. also We conclude 
that M[2,i-q] (xi, a;2, • • • ,x„) = Mfj^^] (xi, X2, • ' ' jS^n), a fo-ct that yields I^^ ''(p) = /''^''(p) = 
H^-^\li) and Df'\p\\v) = D'"^(p||r) = Z),^-^^(p||r). 

3 Jeffreys and Jensen- Shannon type divergences 
3.1 Tsallis type divergences 

We firstly review tlie definitions of two famous divergences. 
Definition 3.1 ([8]. |13j ) The Jeffreys divergence is defined by 

Ji(p||r) = Z)i(p||r) + Z)i(r||p) (12) 

and the Jensen- Shannon divergence is defined as 

..,pi,)4„.(p||E±i)4«.(,|E±i), m 

Analogously we may define the following divergences. 
Definition 3.2 The Jeffreys-Tsallis divergence is 

J,(p||r) = D,(p||r) + L»,(r||p) (14) 
and the Jensen-Shannon-Tsallis divergence is 

JS,.(p||r)4«,(p||£f;)+iD.(r||£f;). (15) 

We find that Jr(p||r) = J,.(r||p) and JSr(p||r) = JSr{r\\p). That is, these divergences are 
symmetric in the above sense. 

To show one of main results in this paper, we need the following lemma that has interest on 
its own. 



1 + expg (-x) 



Lemma 3.3 The function 

f[x) = - \nr 
is concave for < r < q. 

Proof: The proof is a straightforward computation. The second derivative is given by 

f"{x) = -2'^-i{l + (g-l)x}^ fl + {l + (g-l)x}T^^"''~^ 



(g+((/-r){l + (g-l)x}i- 
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Therefore if q > r, then the function f{x) is concave. 

We wish to note here that the above result yields the fact that under the same conditions 
; functic 
affine one. 



the function — In^ jg aigg concave, as the composition of a concave function with an 



Lemma 3.4 Tsallis divergence satisfies 



2 J - 2 — 

Proof: From the famous inequality between the arithmetic and geometric means, we have 



2 

for all j = 1, 2, • • • , n. This implies that 



Pj + 



i=i i=i j=i 2 

1 

-L»i±r(p||r). 



Hence we derive the following result. 
Theorem 3.5 It holds that 



JQ( W I 1 1 + expg (-lj^(pllr)) 1 \ 
Jbr{p\\r) < mm < — m^ , - Ji+r (p||r ) > (16) 



for < r < q. 

Proof: According to Lemma [3? 



JSr{p\\r) = -[-^pjlur ^rjln, 



II ^ l + expglugf^) J}^ 1 + exp„ In, ( 

2^ Pj Inr 



if 1 + exp, ^J^i Pj lug (g) 1 + exp, ^^^1 rj lug 

< - — lUr. lUr- 



2 



1 + expJ--Dg(p||r)) l + expg(-D5(r||p)) 



2 

Then 



In. " -In, -''^ . (17) 



i + expg^^ 2 J_ 1 + exp^ (-ijg(p||r)) 



JS'r(p| |r) < — In, = — In, 

We apply Lemma 13.41 whence it follows 

JSr{p\\r) < \ fl)i+r (p||r) + 2?i+r(r||p)) . 

4 V 2 2 / 

Thus the proof is completed. 
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Remark 3.6 For q = r, r ^ 1 we have JSr{p\\r) — ^5*1, Jr(p||r) — t- Ji and the inequality ([J 
gives us 

TQ ( II I 1 l + exp(-lJi(p||r)) 1 1 
-/S^ilPllr) < mini - log ,-Ji(p||r) S . 

Since 

1 + exp (— x) X 

— log < — 

^ 2 - 2 

we gfei i/ie main results in 

jQ(\\\^ 1 l+exp(-^Ji(p||r)) ^ 1 ^ , ,, , 

^5'i(p||r) < -log <-Ji(p||r). (18) 

3.2 Dual symmetric divergences 

In this subsection, we introduce another type divergences and then we highlight some inequalities 
for them. 

Definition 3.7 The dual symmetric Jeffreys-Tsallis divergence and the dual symmetric Jensen- 
Shannon-Tsallis divergence are defined by 

4'^«)(p||r) = Dr{p\\r) + D2-r(r||p) (19) 

respectively 



(20) 



As one can see directly from the definition, we find that Jr^'^\p\\r) = J2l*^^(r||p) and 

J5r'^^^(p||r) = J52l*J(r||p). See [22] and references therein for additive duality r -H- 2 — r 
in Tsallis statistics. 

Then we get the following upper bound for JSr'^^\p\\r). 

Proposition 3.8 For < r < 2, we have 

J5('^^)(p||r)<ijlg(p||r). (21) 

4 2 

Proof: We infer from Lemma 13.41 that 

D2-rip\\^)<lDs_,{p\\r). 

Consequently 

J5('^^)(p||r) < i (Z)i±,(p||r) + Z)3_,(p||r)) = ^jl£(p||r). 
This completes the proof. 

■ 

In order to derive further results regarding the dual symmetric divercences, we need the 
following lemmas. 

Lemma 3.9 The function exp x is monotone increasing in q, for x > 0. 
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Proof: We have 

dexpgX _ {1 + (1 _ q)x}^ hq{x) 



dq (1 - g)2 

where 

hg{x) = {q- l)x + {1 + (1 - q)x} log {1 + (1 - q)x} . 

Then 

dhg{x) 



dx 



(l-(?)log{l + (l-g)x} >0 



for X > and q> 0. Therefore hq{x) > hq{0) = 0. Thus we have — jj^^ > 0, as asserted. 

■ 

Lemma 3.10 For 1 < r < 2 and x > 0, we have 

— In2_r X < — lUr X 

and 

exp2_r X < exp^ X. 

Proof: Since we have x^~^ + x^~^ > 2, which imphes x^~^ — 1 > 1 — x^~^, we have for 
1 < r < 2, 

x^'i - 1 1 - x^--" , 

m2_r X = > = In^ X. 

r — 1 r — 1 

The second inequahty is a consequence of Lemma 13.91 ■ 
Our next result reads as follows. 

Theorem 3.11 The following inequality holds 

max{j5('^^)(r||p), J5('^^)(p||r)} < - In / + ""^^ (-^(Plk)) ^ (22) 

for all 1 < r < 2 and r < q. 

Proof: By Jensen's inequality, applying Lemma |3.3| we have 



JS^^ >{p\\r) = -l-2^pjlnr 2^rjln2_^ 

^ 1/ l + exp^(-Zj^(p||r)) l + expg(-Z)g(r||p)) 

- 2 V 2 "^""^ 2 

Thus, via Lemma 13.101 it turns out that 

J5(^^)(p||r) < 1 (- m. ' + ^"P^ (-^.(Pll-)) _ + ^-P. (-^.(^Ilp)) 

^ l+expj-|jg(p||r)) 

Further we also have < 2 — r < 1 and the computation is similar for JS'2'i''^(p||r), hence we 
get (using the additive duality) 

J5(-)(r||p) = JStlMr) < _i,^i±^^Mzi^M^. 
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Remark 3.12 For q = r, r — > 1 we have JSr * (p||r) — ^ JS[ (p||r) = JS'i(p||r), (p||r) 
J-['^*^(p||r) = Ji(p||r) and the inequality \2^) yields again the left side inequality of 

Remark 3.13 The inequality ^22\) does not hold for < r < q < 1, in general. We have the 
following counter-example. We consider the probability distributions p = {2/5,2/5,1/5} and 
r = {1/10, 1/10, 4/5}. Then for r = q = 0.1, we have 



In 



l + exp,(-lj,(p||r)) _ ^ 



2 

Open problem 3.14 Prove, disprove or find conditions such that the following inequality holds: 

l + exp,(-lj('^^)(p||r)) 
J5('^^)(p||r)<-ln, ^— ^, re[0,2]\{l}. (23) 



We have not yet found any counter-example of i23\) . One may try to follow the same argument 
as in the proof of Theorem \3.11\ This means that one should prove 

1/ 1 + exp,(-£),(p||r)) 1 + exp2_,(-£'2-r(r||p)) 

2 V"'' 2 2 

1 + exp, (-ij('^^)(p||r)) 
< " K. ^ ^. (24) 

For Q < r < \, we have considered already over 100 particular cases without finding any counter- 
example for Ii24\ ). For 1 < r < 2 we have the following counter-example. Assume r = 1.3, 
p = {0.14, 0.01, 0.85} and r = {0.07, 0.48, 0.45}. Then the right hand side in (2^ minus the left 
hand side in ^2^ approximately equals —0.0125861. Therefore in the case o/l < r < 2 the proof 
of i23\) (if it holds) couldn't begin with Jensen's inequality as a first step. 

3.3 More quasilinear divergences 

We generalize the above definitions. 

Definition 3.15 Let the quasilinear Jeffreys-Tsallis divergence be 

jt{p\\T)^Dt{p\\v) + Dt{v\\v), 
respectively the quasilinear J ens en- Shannon- Tsallis divergence be 



JS*{p||r)si 



o*(pl|!^) + o*(r"'' + '' 



2 ' ' ^ " 2 

The above quasihnear divergences are symmetric in the sense that we have Jr^(p||r) = 
J^(r||p) and JSf{p\\r) = JSf{r\\p). For ip{x) = we obtain J^^ ''(p||r) = Jr.(p||r) and 
J5-'"'(p||r) = J5,(p||r). 

Proposition 3.16 Let ip be a continuous and strictly monotonic function on (0, oo). Suppose 
that ip { ^^^2 ^'^'' ) concave. Then 



1 + exp (-ijg^(p||r) 
J5,^(p||r) < - In, 



for all < r < q. 
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Proof: Since 



1 (JL /i + v"MV' 

J5^;^(p||r)^--ln,^-M^p,V 



1 



by Jensen's inequality, due to the monotonicity and from Lemma 13.31 we just compute 



JSf ip\\r) < M - In, ^ \ ^ '^^ - In. ^ ' 



2 



W l+expj-l),^(p||r)) l + expJ-Z)g^(r||p) 

-I -In. ^-In, 



< — In. 



1 + exp^ (^-ij^(p| 



4 Fermi-Dirac and Bose-Einstein type divergences 

As one-parameter extension of Fermi-Dirac entropy and Bose-Einstein entropy (see also |lltl26j). 
that is of 

n n 

j=i j=i 

and 

n n 

iF^'ip) = -Y.P^ ^°^Pi + log(l 
i=i j=i 

the Fermi-Dirac-Tsallis entropy was introduced in ^26j. Similarly, we may define the Bose- 
Einstein-Tsallis entropy. 

Definition 4.1 The Fermi-Dirac-Tsallis entropy is given by 

n ^ n ^ 

I^^ip) ^ - + E (1 - Pj) (25) 

i=i P^ j=i P^ 

and the Bose-Einstein- Tsallis entropy is defined as 

n ^ n ^ 

^(P) - J2p, - - E (1 + P^) — -. (26) 

Based on the above extensions, we may introduce Fermi-Dirac-Tsallis divergence and Bose- 
Einstein-Tsallis divergence in the following way. 
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Definition 4.2 Let 



3=1 



^(p||r) = - ^p,. m. ^ - (1 - p,) In. J 



Pj 



and 



Z?f^(p||r) = ^ +^ (1 +p,)K, ^ 

Pj ■ ' ^ 



1 + r ,• 



+ Pj 



(27) 



(28) 



T/ien is called the Fermi-Dirac-Tsallis divergence and D^^ is called the Bose-Einstein- 

Tsallis divergence. 



Lemma 4.3 For < x,y < 1 we have 

y 



X lUr 



1-v 4'" 



,,r+l 



y 



r+1 



X 1 — X r + lL 

Proof: Following the idea of O Lemma 11.6.1] we denote 



/ ix,y) = -xln, 
We get 



y 



(1 — x) In, 



1 — X r + 1 



r+l 



y 



x^ (I — x) 



:i-x)" + x"(l-y)"+^-x"(l-x) 



> 0. (29) 



d/ ix,y) 
dy 



y'-ii-yf 



[y'-il-xY-x^il-yY]. 



We can easily check that y(^i_y) ^ 4 under the assumption < y < 1. For y < x (which implies 
y {1 — x) < X {1 — y)) we establish that the function / is decreasing in its second variable, hence 



/ {x, y) > f {x, x) = 0. Clearly for the case y > x we have similarly — ^ 
again f{x,y) > /(x,x) = 0. 
Our next step is to take 

g{x, y) = 2/^+1 (1 - xY + x^ {1 - yY+' - (1 - xY • 

For y > X we may write that 

dc/ {x, y) 



> 0, which leads 



dy 



{r + l)[f{l-xY -x'{l-yY]>^- 



Therefore g{x, y) > g{x, x) = 0. For the case of y < x, one can show that g{x, y) > by the 
similar way. 



Proposition 4.4 The Fermi-Dirac-Tsallis divergence satisfies 

Drip\\r) >^f. [^r' (1 - p^y+p'^ - ^-^y^' -p^j - p^y 



> 0. 



(30) 



Proof: Via Lemma putting x = pj and y = rj, then taking the sum on both sides, it 
follows the claimed result. 



Lemma 4.5 For < x,y < 1 we have 

-xln. - + (l+x)ln, > 

X 1 + X 2^ (r + 1) 



l+x)''y'^+^-(l+y)"+'x" + (l + x)'^x'- > 0. (31) 



\r+l r 
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Proof: Consider the function 

fix, y) = -X K ^ + (1 + x) K, - f (1 + xY - (1 + vY^^ x' + {l + xY x^ 

X 1+ X 2*^ (r + Ij L 

Differentiating / yields 

1 1 



— = (1 + x) y -(1 + y) X 

dy 



y^{l + yY 2'" 



we liave '^■^^^^'^^ < so that /(x, y) > f{x, x) = 0. One checks that for x < y we get ^-^^^'^^ > 0, 
so that f{x,y) > /(x,x) = 0. 



Obviously one has y{l + y) < 2 provided < y < 1. For the case x > y (i.e. x{l + y) > y(l + x)), 
have '^^^Y'^^ < 

ay — 

'hat f{x,y) > 
Next, we put 

9{ 

dg (x, y) 



g{x, y) = {l + xY y'^^ - (1 + 2/)"+' x'' + (1 + x)^ x\ 

For y > X 

(r + 1) [y^{l + xY - + yY] > 0, 



dy 

whence g{x, y) > g{x, x) = 0. Further, for the case y < x, one can prove similarly that g{x, y) > 
holds, which ends the proof. 

■ 

Proposition 4.6 The Bose-Einstein-Tsallis divergence satisfies 

^f^(Pllr) >yj^^t.[(^+P^y^T" - (^ + r,Y^'Pj + (1 +P^yPj] > 0- (32) 

Proof: According to Lemma |4.5^ putting x = pj and y = rj, then taking the sum on both 
sides, it follows the claimed result. 



Remark 4.7 Proposition ^.^] and Proposition \4- 6\ give refined lower bounds for the Fermi-Dirac- 
Tsallis divergence and the Bose-Einstein-Tsallis divergence, respectively. At the same time, they 
assure the nonnegativity of D^^{p\\r) and D^^(p||r). Tsus we easily find that the following 
inequality for the Tsallis relative entropy holds 

{" l-r " 1 + r- 

V (1 - pj) lur ^, - V (1 + Pj) Irir -— ^ 



Corollary 4.8 The following inequalities hold 

n 

'S 



Dnp\\r)>2^{p, 



and 

n 

Df^ip\\r)>-^ip,-r,Y. 



4 



Here Df^{p\\r) is called the Fermi-Dirac divergence, respectively -D^^(p||r) is called the Bose- 
Einstein divergence and their definition corresponds to the limit r — >• 1 in Definition\4.1\ 



Proof: Put r — )• 1 in Proposition 14.41 and Proposition 
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5 Young's inequality and Tsallis entropies with finite sum 

We establish more inequalities involving Tsallis entropy and Tsallis relative entropy applying 
Young's inequality. 

Lemma 5.1 (Young's inequality) Let m, n > and p, (7 G M such that | + | = 1- V P < 
(then < q < 1) or < p < 1 (then q < 0), then one has ^ + ^ < mn. 

Lemma 5.2 (i) Letp,q G M satisfying jh^ + j^ = 1- UP > 1 andO < q < 1, orifO <p <1 
and q > 1, then 

lup X + liig y < xy — 1. 

(ii) Let p,q eR satisfying + = 1. If p < 1 and 1 < q < 2, or if 1 < p < 2 and q < 1, 
then 

1 1 

hip — h ln„ - > —xy + 1. 
X ^ y 

Proof : 

(i) Using Lemma |5.H we obtain 

x^-P-1 y^-'^-l 

hip x + lugy = — \ <xy - I. 

I — p 1 — 9 

(ii) Lemma l5 . II leads to 



1 , 1 xP'^ - 1 yi-^ - 1 / xP-i yi-^ \ f 1 1 

-+\ng- = — + — = - + + + 

X y 1—p 1 — 9 \P — ^ Q — ^J \P — ^ Q — 



Then we have the following proposition. 



if < p < 1 and 1 < q < 2, then 



Proposition 5.3 (i) Let p,q £ satisfying + = 1. If 1 < p < 2 and < q < 1, or 



Dp{p\\r) + H2-g{p) > 1 - ^PjTj (33) 



and 



n 



I)2-p(p||r) + F,(p)<^^-l. (34) 

i=i 

(ii) Let p, (7 G M satisfying + = 1. If < p < 1 and l<q<2orifl<p<2 and 
< q < 1, then 

n 

Z)p(p||r) + /f2-,(p)<^^-l (35) 

and 

n 

D2-p(p||r) + Hg{p) > 1 - Y^PjVj. (36) 

i=i 

Proof : 
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(i) In (i) of Lemma 15.21 since we have In^ y = — ln2-q | for all y > 0, we get 

Inp X + ln„ y = ln„ X - ln2_„ - < xy - 1. 

y 

Putting x = p- and y = pj and multiplying —pj and then taking the sum on both sides, it 
follows 

n ^ n \ ^ 

- Yl Pj ^"^P^ + Yl Pj —^Yl ^Pi ~ Pj^'j ) ' 

3=1 P^ j=l P^ j=l 

which implies the inequality (|33p. We also have the inequality ()34p from 

lup X + lnqy = - ln2_p ^ + In^ y < - 1. 

(ii) Using (ii) of Lemma 15.21 we have two inequalities ()35p and ()36p by the similar way to the 
proof of (i). 



Remark 5.4 We have a pair of additive duality (p, 2 — g) -H- (2 — p,q) between (i) and (ii) of 
Proposition 15.31 



A cross-entropy type formula [15j of two probability distributions is the following: 

/?(p,r) = I)i(p||r) + ifi(p). 
One may see the left side terms in Proposition 15.31 as some generalizations of -ff(p, r). 
Corollary 5.5 The following inequalities holds: 

n n 

0<l-^p^.r,.<i7(p,r)<^^-L 
i=i 

Proof: In Proposition 15.31 we take g — >• 1. 



j=l j=l 



Corollary 5.6 The following inequalities hold: 



< 1 - ^p] < Hi{p) <n-l. 



Proof: In Corollary 15. 5^ we take r = p. 



Proposition 5.7 Let p,q (zM. satisfying + = 1. Ifp < 1 and 1 < q < 2, orifl<p<2 
and q < I, then 



/;^(p) + /f^(p)>3^p,(l-p,) 
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Proof: From Lemma 15.21 (ii), putting x = y = pj and multiplying pj and then taking the 
sum on both sides, it follows 



n 1 " 1 " 

j=l f3 j=l t'3 -^^ 

Putting X = y = 1 — Pj and multiplying 1 — pj and then taking the sum on both sides, it follows 

n ^71 ^ n 

^(l-p,)lnp^— -+j;(l-p,)ln,^— ->-^(l-p,)3 + n-l. 

j=i j=i j=i 

Summing up these two inequalities we get 



j=l j=l j=l 

■ 

We also find that the following interesting inequalities on finite sum hold true. 

Proposition 5.8 For two probability distributions p = {pi,p2, ' ,Pn} andr = {ri,r2, ■ ■ ■ ,rn}, 
we have the following relations. 

n q l—q ^ -\ ^ Tr^n 2 — q q—l 



(i) If ^ < q < ^, then we have X]j=iPj^j < 1 < Yl]=iPj 



r- 



(ii) If 1 < q <2, then we have Yl]=iP'j''^j > 1 > Yl]=iPj '^"'^ ^ 



Proof: From the nonnegativity of Tsallis relative entropy Dq(p||r) > and L'2-ij(p||r) > 0, 
we have the statements. 



6 Concluding remarks 

We close this paper giving further generalized entropy and divergence by the use of two- 
parameter extended logarithmic function. 

Definition 6.1 For a continuous and strictly monotonic function ip on (0, oo) and r,q >0 with 
r,q ^1, the {r,q)-quasilinear entropy is defined by 



(37) 



Here the two-parameter extended logarithmic function [19 1 is given by ln,.^q (x) = In^ exp In,, (x) 
Correspondingly, the inverse function of In^^g is denoted by exp^ ^ . For tp{x) = In^^g (x) we recover 
the entropy used in [191 Section 4]. 

For 'ip{x) = x^"*", we have an extension of Tsallis entropy 

1—?' 

Ir,q (P) = exp Hr{p) = Hr,q{p). 

For ip{x) = x^~P, we have 

4^;"''(p) = ln,expi?,^-^(p) = Hr,g,p{p) 
that extends Sharma-Mittal entropy to a three-parameter entropy. 
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Definition 6.2 For a continuous and strictly monotonic function ip on (0, oo) and r,q >0 with 
r,q ^1, the {r,q)-quasilinear divergence is defined by 



(38) 



For 'ip{x) = X ^, we get the following extension of Tsallis relative entropy 

1—?' 

D^g (p||r) = ln5expDr(p||r) = Dr,q{p\\r). 

For ip{x) = x^~P, we have 

D^^'^ipWr) = lngexpi:>^p^^(p||r) = Dr,q,p{p\\r) 

that extends Sharma-Mittal divergence to a three-parameter divergence. 

For a three parametrization extension of the logarithmic function see for instance [12] and 
the references cited therein. With such extensions the quasihnear entropies can be analogously 
extended to three parametric classes too. This is not the purpose of the present paper. 
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